1. Field of the Invention
This invention relates to image processing.
2. Description of the Related Art
An example image processing application will now be described in connection with a digital video camera.
In order to generate a colour image using a solid state image sensor (such as a charge coupled device (CCD) image sensor) it is necessary to ensure that different colour components are detected in respect of each pixel position. Some arrangement of colour filters is needed because typically the individual photosensors in an image sensor detect light intensity with little or no wavelength specificity, and therefore cannot themselves detect separate colour information.
One way to provide colour detection is to employ multiple image sensors, each detecting a respective colour component. For example, many digital video cameras use a three CCD system, with each CCD detecting a respective one of green, red and blue colour components. This arrangement requires the three CCDs to be optically aligned with one another to a sub-pixel accuracy, and also needs a light splitter and colour filter arrangement. Optical filters such as dichroic (or thin film) filters may be used for this task, as they have the useful property of passing light of a range of wavelengths while deflecting (rather than absorbing) light outside that range.
However, many digital cameras instead make use of a single CCD operating in conjunction with a colour filter array (CFA). The CFA is a mosaic of tiny colour filters fabricated over the individual pixel sensors of the image sensor.
The colour filters filter the light by wavelength range, such that the separate filtered intensities include information about the colour of light. Absorption (rather than dichroic) filters are generally used.
An example of the use of a CFA is shown schematically in FIG. 1 of the accompanying drawings. A scene 10 is captured by a camera 90 comprising a lens arrangement 20, a CFA 30 arranged on a CCD image sensor 40, and an image processor 50. The CFA 30 comprises an array of individual filter elements 80, generally one per individual pixel sensor of the CCD image sensor 40, each arranged to pass light of a respective colour. Although in FIG. 1 the CFA 30 is shown as an overlay on the CCD image sensor 40, in practice the optical filter elements can be fabricated directly on top of the pixels sensors as part of an overall semiconductor fabrication process.
There are many types of CFA. For example, a so-called Bayer filter, originally proposed in U.S. Pat. No. 3,971,065, contains filter elements giving information about the intensity of light in red, green, and blue (RGB) wavelength regions. An example of the pixel filter layout of a Bayer filter is shown in FIG. 2 of the accompanying drawings, which schematically illustrates the filter layout across a small number of pixel positions 100, with R representing a filter which passes red light, G representing a filter which passes green light, and B representing a filter which passes blue light. A basic pattern of four pixels (a square array of 2×2 pixels) 110 is repeated across the full extent of the CCD sensor.
One reason that more green elements are provided than red or blue elements is that the human eye is generally more sensitive to green light than to the other primary colours. So, the additional resolution in respect of green light mimics the colour response of the human eye. It is also useful because when red, green and blue colour component signals are electrically combined together to form a monochromatic luminance signal (Y), contributions from each colour component are weighted so that the luminance signal contains a greater proportion of the green component than either of the other two components.
The raw image data captured by the image sensor is then converted to the full-colour image, with intensities of all three primary colours represented at each pixel, by a so-called demosaicing algorithm which is tailored for each type of colour filter. In other words, the mosaic relationship of the detected colour components is undone by the demosaicing algorithm so as to provide, for each pixel, all three colour components in respect of a single effective pixel position. In the system of FIG. 1 this function is provided by the image processor 50, which generates an output colour image 60. The spectral transmittance of the CFA elements along with the demosaicing algorithm jointly determine the colour rendition of the camera.